A Multivalued Representation for View Synthesis

We propose a new depth-based representation of 3-D scenes primarily for the problem of arbitrary view synthesis. The information contained in a given image sequence is com-pacted into a single multivalued array, where points are organized by occlusions rather than by coherent aane motions. This grouping facilitates an automatic process to determine the number of layers and helps to reduce the ar-tifacts caused by occlusions in the scene. In addition, an iterative multiframe dynamic programming algorithm is described to produce piecewise smooth depth maps. A novel multiframe segmentation, tracking, and plane tting algorithm is also proposed to handle the traditionally diicult low-contrast regions. Reconstructed views as well as arbitrary yarounds of real scenes are presented to demonstrate the eeectiveness of the approach.